inscatolati - Get out of the box! :)

In kernel 2.4 and first 2.6, raid partitions aka md devices were not partitionable. Even today, most distributions do not support md device partitioning.

There are not many solutions to this problem, either:

• create a single raid device for the whole disk (for example, between /dev/sda and /dev/sdb), create a file system on that single partition, and live with it (mkfs.ext3 /dev/md0, where md0 is made of /dev/sda and /dev/sdb). This should have no problems, beshide that it is not very sane to put a complete system on a single partition...

• create a single raid device for the whole disk (for example, between /dev/sda and /dev/sdb), and use the device mapper (aka lvm) to create logical volumes (aka, partitions). This works, but you cannot boot from a logical volume... neither lilo nor grub support that.

• first create the partitions, and then create the raid devices. For example, you could create a small /dev/sda1 and /dev/sdb1, and a md0 device with the /boot directory on it. Then, create /dev/sda2, /dev/sdb2, /dev/sda3, /dev/sdb3 as needed and raid them separately... this solution works well, but:

  • you have to pay attention when installing the boot loader. Since only partitions are on 'raid', everything out of the partitions, like the MBR (Master boot record) is not in 'raid'. If you simply install grub or lilo as usual, at the first failure, after replacing a disk, you will likely end up with an unbootable system.

  • the solution is not very flexible. If you need to create a new partition, you have to create a new raid device, and so on... if you need to change something, it's hard to do...

  • in case of a disk failure, you will probably end up with many software raid devices not working anymore, and with the kernel having to realize that all or some of them have failed...

• Well, simply create two partitions, /dev/sda1 and /dev/sda2, raid /dev/sda1 with /dev/sdb1, and then use /dev/sda2 in raid with /dev/sdb2 as an phisica volume for LVM. That way, you can boot the system, and create new partitions simply by using the LVM.

With kernel 2.6 and a recent version of mdadm, you can also use the option --auto=mdp to create a partitionable raid volume. One more solution could be to use the EVMS, still not part of the official linux kernel.

As you can read on the Linux Software RAID HOWTO, you should set the type of raid partitions to 0xfd. Note, however, that there are two ways to assemble raid devices:

• asking the kernel to do it automatically, at boot time.

• by running mdadm or the raidhot tools right at boot time, telling them to assemble raid devices.

In practice: if you put 0xfd in the type flag of the partition table, the device will be automatically assembled at boot time. If you don't, you will need to configure mdadm and/or raidhot tools to do that for you (at boot time), or you will have to assemble the device manually.

To create the mdadm configuration file, you have two choices: one, create it manually, two, run

# mdadm --examine --scan > mdadm.conf or # mdadm --detail --scan > mdadm.conf

(the two commands give the same output, sometimes in a different order). These commands require your raid devices to be active and available. If they are not active, you can either assemble them manually, and run the above commands, or use the command:

# mdadm --examine --brief --scan --config=partitions > mdadm.conf

The standard location for the mdadm.conf file, on Debian systems, is /etc/mdadm/mdadm.conf.

Ok, let's say you have a software RAID 1 /dev/md0 device made of two partitions on two different scsi disks, /dev/sda1 and /dev/sdb1.

Let's say you just had a major hardware failure, and for one reason or another, some data was corrupted on the first device, while some other data was corrupted on the second device.

One easy way to try to recover data is to mount /dev/sda1 and /dev/sdb1 individually, recover data, and then, eventually, put them back into the raid.

Doing so, is quite easy:

• if the raid device is still mounted, umount it immediately, with something like 'umount /mount/point/of/raid/device'. You can see the mount point of the md device with something like 'mount |grep md0' or 'cat /proc/mounts |grep md0' (see note mount, read only partitions, and wierd output).

• stop the raid device, with something like:

  # mdadm --misc --stop /dev/md0

  or mark it read only, with:

  # mdadm --misc --readonly /dev/md0

• simply mount the devices independently, using:

  # mount -o ro /dev/sda1 /mnt/sda1 # mount -o ro /dev/sda2 /mnt/sda2

  making sure the /mnt/sda1 and /mnt/sda2 directory exist.

• You can now work on /mnt/sda1 or /mnt/sda2 without problems (...).

• if you later decide that the content of one of the partitions is good, you can start the array in degraded mode with --assemble, or mark /dev/sda2 (for example) as failed, and then simply replace it ...